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Abstract:  When  the  endoscopist  is  exploring  the  digestive  cavity, 
the  action  of  focusing  on  a  particular  organ  area  and  of  evoking 
diagnostic  hypotheses  underlies  a  complex  march  of  the  thought. 
This  mind  process  has  to  be  analyzed  in  terms  of  data  fusion  and 
of  reasoning  in  order  to  lay  foundations  with  a  view  to  a 
computerized  expert  system  in  endoscopy.  Provided  with  evolved 
functionalities  (the  similar  case  retrieval  and  the  diagnostic 
hypothesis  evaluation),  a  digestive  endoscopy  atlas  can  be  used  as 
a  tool  of  training  and  even  of  diagnosis  aid  for  “junior” 
endoscopists.  The  architecture  of  such  a  system  is  to  have  its 
grounds  on  a  Case  Based  Reasoning  (CBR),  organized  around 
two  bases,  one  of  endoscopic  knowledge  and  one  of  case 
iconography. 

The  retrieval  of  similar  cases  can  be  assimilated  to  a  fusion 
process  with  two  steps,  classification  (global  method)  owing  to 
the  knowledge  base  and,  afterwards,  measurement  (local 
method)  by  means  of  selected  cases  of  the  endoscopic  case  base. 
Lesion  identification  and  Disease  recognition  -  that  is,  the  Scene 
analyze  -  are  the  driving  forces  not  only  in  the  Classification 
stage,  but  also  in  the  Similarity  measure  stage.  Besides, 
classification  algorithms,  tested  on  simulated  endoscopic 
descriptions,  afford  results  full  of  promise. 

Keywords  -  Upper  Digestive  Endoscopy,  Data  Fusion,  Case-Based 
Reasoning,  Approximate  Reasoning,  Classification. 

I.  Introduction 

The  conceiving  of  a  knowledge  based  inference 
system  for  help  to  decision  in  the  upper  digestive  endoscopy 
implies  a  narrow  collaboration  between  gastro-enterology 
physicians  and  information  processing  specialists,  so  as  to 
adapt  the  medical  reasoning  in  the  computer  world. 

The  project  intends  to  be  a  computerized  advanced 
adaptation  of  medical  atlas-books.  Next  to  a  classic 
illustration  of  diagnoses  by  means  of  noteworthy  iconography, 
such  an  atlas  must,  in  addition,  integrate  the  similar  case 
retrieval  and  the  diagnostic  hypothesis  evaluation.  For  the 
first  point,  the  general  idea  is  that,  faced  with  a  situation  he 
can’t  interpret,  any  user  is  nevertheless  able  to  describe  its 
content  and  also  that,  with  an  image  database  system  which 
includes  content-based  indexing ,  storage  and  retrieval ,  all  the 
similar  situations  can  be  proposed  with  their  interpretations  in 
order  to  aid  him.  The  second  point  corresponds  to  the  case  in 
which  the  user  describes  an  endoscopic  scene  and  puts 
forward  a  diagnostic;  the  system  then  supplies  a  hypothesis 
evaluation  as  well  as  similar  scenes  with  similar  or  differential 
diagnoses. 


The  following  section  briefly  describes  the  upper  digestive 
endoscopy  domain  and  the  physician  diagnostic  approach.  In 
section  3,  the  project  is  depicted  through  its  aims  and  its 
general  architecture.  Next,  section  4,  the  information  in 
endoscopy,  particularly  the  description  as  well  of  pathologies 
as  of  patient's  exam  iconography,  is  drawn  up  thanks  to  the 
scene/object  modeling.  In  section  5,  the  scene  analyze,  the 
matter  of  the  similar  case  retrieval,  is  sketched  through  a  two- 
leveled  fusion  framework.  Ensuing  statistical  tests,  already, 
present  promising  results.  In  conclusion,  further  steps  of 
research  development  will  be  suggested. 

II.  Endoscopic  world 

Endoscopic  examination  of  the  upper  gastrointestinal  tract 
-  esophagus,  stomach  and  duodenum  -  is  the  most  accurate 
and  informative  method  that  may  evaluate  a  wide  variety  of 
digestive  symptoms  (see  Figure  1).  Since  its  introduction  for 
widespread  use  in  the  late  1960's,  the  technique  of  endoscopy 
has  expanded  the  understanding  of  numerous  gastrointestinal 
diseases  thanks  to  a  careful  inspection  of  the  mucosal  surface 
and  thus  has  greatly  improved  ability  to  care  affected  patients. 
Since  the  end  of  the  80's,  videoendoscopes,  which  incorporate 
miniature  TV  cameras  at  their  tips,  allow  the  image 
transmission  for  video  display,  storage  and  analysis. 


Figure  1:  l.a.  Several  endoscopic  views 
l.b.  Visited  organs 
l.c.  The  endoscope 
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Physician’s  reasoning  [1]  in  endoscopy  emphasizes,  in  the 
diagnostic  process,  two  decision  levels  which  refer  to  two 
information  spaces:  the  endoscopical  findings,  i.e.  the  lesions, 
and  the  diseases.  An  interaction  connects  these  two  decision 
levels  because  the  diagnosis  of  endoscopical  findings  meddles 
with  the  disease  diagnostic  decision.  Inconsistencies  in  the 
final  decision  according  to  other  information  (medical 
context,  other  endoscopical  findings)  must  lead  to  doubt  about 
the  validity  of  endoscopical  finding  diagnoses.  At  the  disease 
level,  the  decision  of  assigning  a  diagnostic  class  also  depends 
on  the  prevalence  of  this  diagnosis  in  the  current  practice  and 
on  the  endoscopist  experience  for  rare  cases. 

The  analysis  of  the  medical  context  leads  to  distinguish  two 
reasoning  approaches.  In  one  hand,  the  medical  context  is  not 
specific.  A  systematic  exploration  of  the  organ  permits  to 
focus  on  lesions,  according  to  abnormal  variations  of  color, 
relief  or  anatomical  repairs.  Medical  knowledge  of  lesion 
characteristics  leads  to  the  diagnosis  of  these  elementary 
objects.  The  association  of  lesions  compared  with  the 
knowledge  of  the  endoscopic  aspects  of  diseases  induces  to 
the  generating  of  diagnostic  hypothesis.  In  the  other  hand,  a 
specific  context  earlier  prompts  to  formulate  diagnostic 
hypothesis  and  to  consider  endoscopic  lesions  whose 
association  would  confirm  the  diagnosis.  Moreover,  the 
endoscopist  closely  explores  the  regions  where  these  lesions 
are  usually  found.  During  endoscopic  examination,  the  two 
schematic  approaches  are  not  exclusive. 

III.  Diagnosis  Aid  in  Endoscopy 

First  considering  the  system  development  and  its 
capabilities,  then  the  Case  Based  Reasoning  (CBR)  [2]  is  to  be 
sketched  as  leaning  on  both  a  Case  Base  and  a  Knowledge 
Base  (those  to  be  detailed  further). 

A)  General  approach 

For  the  present,  with  respect  to  the  endoscopic  imagery, 
there  is  no  question  of  exploiting  the  numerical  content  of 
images  or  sequences  because  such  a  content  would  not  be 
sufficient  to  "translate"  the  medical  meaning  of  the  situation. 
By  this  last  term  must  be  understood  the  patient  context  and 
the  scene  concept,  i.e.  the  anatomical  location,  as  well  as  any 
information,  carried  or  not  carried  by  the  imagery,  the  whole 
guiding  the  doctor  in  his  diagnostic  interpretation.  That  is  why 
the  followed  approach  consists  in  elaborating  a  model  of  the 
situations  encountered  with  the  imagery,  that  is  to  say  a 
knowledge  base  of  endoscopic  diagnoses  exemplified  by  an 
iconographic  base.  This  last  one  is  constituted  by  a  set  of 
cases,  i.e.  images  and/or  sequences  acquired  from  endoscopic 
examinations  (stored  on  videotape)  and  illustrated  by  their 
diagnostic  reports. 

B)  System  capabilities 
Classical  retrieval 

With  this  use  mode,  it  is  about  functionalities  given  by  a 
database  management  system;  in  other  words,  the  retrieval 
according  to  a  request  or  a  logical  combination  of  requests 


Similar  case  retrieval 

If  a  user  describes  a  lesion,  the  images  or  sequences 
showing  similar  lesions  and  the  related  endoscopic  reports  are 
provided.  Otherwise,  if  a  user  describes  a  patient's  exam,  i.e. 
the  reasons  of  the  endoscopy  and  the  lesions,  the  system 
proposes  the  potential  diagnosis(es)  of  disease(s)  and  images 
or  sequences  of  similar  cases. 

Diagnostic  hypothesis  evaluation 

The  user  describes  a  lesion  and  proposes  a  diagnostic  about 
it;  the  system  supplies  a  plausibility  evaluation  and  shows 
similar  lesions  with  similar  or  differential  diagnoses. 
Otherwise,  the  user  describes  the  patient's  exam  and  proposes 
the  diagnoses  of  the  lesions  and  of  the  diseases;  the  system 
returns  a  likelihood  of  the  situation  and  shows  images  and 
diagnoses  which  can  confirm  or  contradict  the  user  analyze. 

C)  Endoscopic  CBR 

As  a  physician  exploits  his  medical  knowledge  (anatomy, 
physiology,  epidemiology  and  pathology)  and  his  experience 
(previously-encountered  patient  cases)  in  order  to  work  out  a 
diagnosis  of  the  diseases,  the  system  will  be  based  on  two 
bases,  one  of  endoscopic  semiology  (disease  descriptions)  and 
one  of  endoscopic  exams  (patients'  iconography). 

A  Case-Based  Reasoning  (CBR)  [2]  whose  philosophy 
depends  on  the  simple  postulate  that  the  experience  gained  in 
solving  problems  (cases)  can,  by  analogy,  permits  to  solve  the 
similar  ones  (new  cases),  is  well  adapted  to  the  diagnosis  aid 
in  endoscopy.  The  base  of  cases  therefore  represents  the 
system  "memory"  when,  thanks  to  the  knowledge  base,  the 
indexing  allows  its  "training". 

The  CBR  cycle  admits  4  steps:  - description ,  retrieval , 
correction  and  training  -  (see  Figure  2).  The  arrow  from 
Retrieval  to  Description  illustrates  the  intention  to  minimize 
the  new  case  description;  an  adaptive  algorithm  will  supply 
the  user  with  a  pertinent  questioning. 


IV.  Scene/Object  Modeling 


Drawn  from  the  Minimal  Standard  Terminology  [3]  of  the 
ESGE  (European  Society  of  Gastro-Enterology),  a  two- 
leveled  description  mode  of  the  endoscopic  imaging  and  of 
the  gastro-enterology  pathologies  is  illustrated  by  the  concept 
of  Scenes  with  Objects. 

A)  Endoscopic  Scenes 

Owing  to  the  federative  concept  of  Scenes  with  Objects,  the 
description  of  endoscopic  information  allows  to  distinguish 
three  types  of  Scenes: 

-  Physical  Scenes 

The  file  of  an  image  or  of  an  image  sequence  is  considered 
to  be  a  Physical  Scene.  It  visualizes  an  interesting  part  of  the 
endoscopical  exam,  showing  anomalies,  that  is  the  Objects. 

-  Logical  Scenes 

A  Logical  Scene  represents  a  medical  interpretation  of 
endoscopic  imagery,  i.e.  an  endoscopic  disease  diagnosis, 
which  associates  a  peculiar  patient  context,  one  or  several 
endoscopical  Finding(s)  or  Object (s)  and  their  eventual  spatial 
relations. 

Conceptual  Scenes 

As  abstractions  of  Logical  Scenes,  Conceptual  Scenes  are 
the  extended  definitions  of  the  upper  digestive  tract 
pathologies.  Patient  context,  reasons  for  the  endoscopy,  one 
or  several  Conceptual  Objects  with  their  eventual  spatial 
relations,  and  the  complementary  procedures  to  be  advised, 
constitute  the  medical  knowledge  of  these  Scenes. 

B)  Object  Information 


Lesions  or  any  element  of  interest,  i.e.  the  "endoscopic 
findings",  constitute  the  objects  to  be  depicted  thanks  to  an 
exhaustive  description  mode  (see  Figure  3).  So,  each  object  is 
described  with  22  features  (even  31  if  a  sub-object  exists).  To 
each  feature  is  associated  a  set  of  choices,  representative  of  all 
possibilities  and  judiciously  defined  by  the  expert. 
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Figure  3:  Object  description  features 

C)  Scene  Information 

At  least  one  object,  possible  spatial  relations  and  the  patient 
profile  represent  the  “how”  and  the  “where”  of  a  diagnosis, 
the  complementary  procedures  to  be  envisaged  denoting  the 
“then”.  As  shown  on  Figure  4,  information  on  disease 
diagnosis  well  encompasses  four  relevant  items: 
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Figure  4:  Disease  associated  Information 


D)  Endoscopic  Knowledge  Base 


As  an  expert  squeezes  out  his  knowledge,  using  linguistic 
valuations  and  instilling  even  doubt,  the  chosen  way  (see 
Figure  5)  is  supposed  to  do  the  same,  manipulating  linguistic 
truth  degrees  as  well  as  uncertainty  or  vagueness  [4] . 
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Figure  5:  Knowledge  Linguistic  Valuation 


An  object  is  systematically  illustrated  upon  140  (or  205) 
modalities  corresponding  to  the  22  (or  31)  features  with  the 
Knowledge  Linguistic  Valuation  and  is  marked  by  ESGE 
code  and  libel.  A  scene  is  depicted  by  a  patient  profile  (the 
sex  and  age  prevalence  features  as  well  as  a  predefined  whole 
of  clinical  contexts),  by  the  objects  (one  of  them  must  be 
Essential),  by  eventual  spatial  relations  between  objects  and 
by  the  complementary  procedures,  all  these  still  with  the 
Knowledge  Linguistic  Valuation.  The  diseases  are  called  by 
an  ESGE  denomination  and  are  classified  as  main  or  not. 

To  sum  up,  object  and  scene  descriptions,  i.e.  the 
Conceptual  Scenes ,  form  a  two-leveled  knowledge  base. 


E)  Endoscopic  Case  Base 


For  the  imagery,  it  is  a  whole  of  endoscopic  examinations, 
which  must  be  indexed.  Constituted  of  images  or  sequences 
and  of  a  diagnostic  report,  each  examination  represents  a  set 
of  Physical  Scenes  as  well  as  a  set  of  Logical  Scenes.  While  a 
Physical  Scene  is  a  file  where  are  visualized  the  objects,  a 
Logical  Scene  represents  an  endoscopic  diagnosis  which 
consists  of  information  concerning  the  patient  (common  to  all 
the  Logical  Scenes  of  a  same  examination)  and  of  all  the 
objects  attached  to  the  diagnosis.  Moreover,  the  indexing  must 
hold  into  account  that  Logical  and  Physical  Scenes  do  not 
tally. 


V. 


Scene  Analyze 


A)  Towards  solving  new  cases 

Usually,  the  similar  case  retrieval  is  based  either  on  a 
Similarity  measure  (computational  approach)  or  on  an 
Indexing  structure  (representational  approach).  But  here, 
these  two  approaches  (see  Figure  6)  are  to  be  combined:  a 
first  step  will  classify  owing  to  the  Indexing  structure  (global 
method)  and,  after,  a  similarity  measure  (local  method)  will 
complete  the  retrieval  of  relevant  cases. 


Figure  6:  similar  case  retrieval  scheme 


Lesion  and  Disease  Classification  are  the  key  points  of  the 
Scene  analyze.  With  this  intent,  the  questioning  interface 
allows  the  user  to  depict  an  endoscopic  exam,  in  other  words 
the  patient  profile,  the  objects  and  possible  spatial  relations 
between  objects.  To  avoid  a  boring  description  of  objects, 
only  5  features  (anatomical  position,  form,  colour,  relief  and 
type)  are  wanted  at  first;  the  other  ones,  according  to  their 
discriminating  power,  will  be  selected  to  refine  the  object 
recognition.  After  the  object  classification,  the  whole 
description  of  the  exam  is  analyzed  to  identify  one  or  several 
Logical  Scenes  -  i.e.  the  diagnoses  of  diseases  -.  This  analysis 
of  scene  still  allows  perfecting  the  classification  of  the  lesions 
-  i.e.  objects-.  Indeed,  as  the  level  of  the  image  (or  of  the 
sequence)  generally  represents  the  endoscopic  lesion  level,  the 
classification  should  especially  insist  on  the  objects  in  order  to 
select  a  subset  of  candidate  cases  (the  similarity  playing  upon 
those),  while,  from  a  medical  point  of  view,  the  level  of 
interest  should  surely  envisage  the  pathology  diagnosis. 

B)  Classification  results 

Object  Classification  Algorithm 

Thanks  to  the  expert,  95  Conceptual  Objects  have  been 
described  on  31  features,  i.e.  on  205  modalities. 

The  Object  Classification  Algorithm  eliminates  owing  to 
impossible  or  excluded  valuation  and  ranks  according  to  the 
Knowledge  Linguistic  Valuation.  For  each  Conceptual 
Object,  300  Logical  Object  descriptions  are  generated  and 
tested  by  means  of  the  algorithm;  Figure  7. a.  shows  a 
"perfect"  classification  of  95%. 
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Figure  7:  Classification  results  on  Object  descriptions 


Discrimination  analyze 

For  each  Logical  Object  description  based  on  22  features  on 
average,  a  minimal  description  exists  on  fewer  features.  From 
where,  anatomical  position,  form,  color,  relief  and  type 
usually  belong  to  the  minimal  descriptions  of  the  95*300 
precedent  tests.  Figure  7.b.  shows  the  Classification  test 
results  with,  at  the  beginning,  a  description  on  the  5  features, 
completed  by  2  others  as  long  as  there  are  discriminating 
features.  Results  anyhow  remain  excellent. 


Scene  Classification  Algorithm 
The  expert  has  described  150  Conceptual  Scenes 
representative  of  75  ESGE  diagnoses. 

The  Scene  Classification  Algorithm  selects  by  essential 
valuation,  eliminates  owing  to  impossible  or  excluded 
valuation  and  ranks  according  to  the  Knowledge  Linguistic 
Valuation.  For  each  Conceptual  Scene,  50  Logical  Scene 
descriptions  are  generated  and  tested  with  the  algorithm; 
Figure  8  shows  a  "perfect"  classification  of  84%. 


Figure  8:  Classification  results  on  Scene  descriptions 


VI.  Concluding 


The  present  paper  has,  step  by  step,  reported  a  laborious 
scrutiny  for  the  knowledge  representation  and  the  case 
indexing  in  digestive  endoscopy.  The  scene  analyze  shows 
cheering  classification  results  -but  those  to  be  still  improved-. 
The  tackling  of  similarity  measure  will  finish  off  the  whole 
system.  Further  ambitions  would  look  to  evolving  towards 
Telemedecine  thanks  to  a  user  interface,  developed  with  Internet 
tools... 
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